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Executive Summary 



The ranking of higher education institutions is a growing phenomenon around the globe, with 
ranking systems in place in more than 40 countries and the emergence of international ranking 
systems that compare institutions across national lines. With this proliferation of rankings come 
questions about the goals, uses, and outcomes of these systems. 



This issue brief seeks to understand the role that rankings 
play in institutional decision making and how institutions 
in various countries use rankings in ways that might benefit 
higher education in the United States. The study is based 
on interviews with campus stakeholders at institutions in 
Australia, Canada, Germany, and Japan — countries that 
have their own national ranking systems as well as a 
presence in the international systems. 

A review of current research suggests that rankings influence 
institutional decision making in the following areas: strategic 
positioning and planning, staffing and organization, quality 
assurance, resource allocation and fundraising, and admissions 
and financial aid. To better understand these issues and how 
rankings affect them, Institute for Higher Education Policy (IHEP) 
discusses the ways by which institutions in the four countries 
incorporate rankings into their decision-making frameworks. 

KEY FINDINGS AND RECOMMENDATIONS 

Rankings impact higher education decision making in the 
profiled countries in many ways that are consistent with research 
on the topic; however, the interviews suggest some nuances and 
unique approaches that could prompt new uses for rankings 
at U.S. institutions. Officials interviewed for this report offered 
many valid criticisms of rankings: that too much importance 
is given to rankings over other institutional assessments and 
practices, emphasizing institutional rankings decreases access 
for underserved students, and rankings skew institutional 
resources to favor research over teaching. However, the findings 
highlight a number of recommendations for how institutions 
can leverage rankings to do their work in innovative ways. 



Improved data-based decision making. Higher education 
institutions, especially those in the United States, are increas- 
ingly called on to use data to inform their decision making and 
to document student and institutional success. Rankings can 
prompt institutional discussions about what constitutes 
success and how the institution can better document and 
report that success. 

Increased participation in broader discussions about 
measuring institutional success. Rankings can encourage 
institutions to move beyond their internal conversations to 
participate in broader national and international discussions 
about new ways of capturing and reporting indicators of 
success. The participation of individual institutions in these 
discussions will become an increasingly important way for 
articulating how rankings can be used to measure and 
improve institutional practices. 

Improved teaching and learning practices. While the 
case study institutions continue to point to their changing 
practices that alter input indicators — increasing selectivity, 
favoring research over teaching, and strengthening the faculty 
profile — a number of institutions are also reporting changes 
to practices directly related to student learning and success. 
Institutions that use their rankings to prompt change in areas 
that directly improve student learning experiences demon- 
strate that rankings can lead to positive change in teaching 
and learning practices. 

Identification and replication of model programs. For some 
institutions in the study, the peer benchmarking function of 
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rankings is leading to identification and replication of model 
programs. Institutions should be open to using rankings to 
identify and share best practices. 

Increased institutional collaboration. Rankings are often 
perceived as instigators of competition among institutions, but 
the case studies suggest that they also foster collaboration, 
such as research partnerships, student and faculty exchange 
programs, and alliances. Rankings can be important starting 
points to identify institutions with which to collaborate and partner. 

In highlighting ways ranking systems can positively impact 
institutional decision making, this issue brief also underscores 
the continued need for attention to potential negative effects 
of rankings. These include the degree to which rankings — and 
an emphasis on developing world-class universities — under- 
mine college access for disadvantaged student populations; 
an unbalanced emphasis on research over teaching; the ratio 
between full-time and adjunct faculty; the improvement of key 
rankings variables as a substitute for comprehensive, institution- 
generated strategic planning; and the funding of world-class 
institutions at the expense of institutions that further other national 
goals. Institutions should consider the concerns raised about the 
effects of rankings as catalysts for direct policy actions to mitigate 
potential negative impacts. 

College rankings are an entrenched phenomenon around the 
world. This brief documents positive and negative institutional 
responses to rankings, and suggests ways institutions can 
leverage them to improve internal and external practices. It 
can be a starting point for institutions that are seeking new 
tools for institutional change and new ways of responding to 
their rankings in national and international systems. ^ 
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Introduction 



The United States is not alone in the phenomenon of college and university rankings. Building on 
the foundation originally established with the U.S. News and World Report ranking of undergraduate 
colleges in the 1980s, numerous magazines, newspapers, organizations, individuals, and government 
agencies around the world publish university rankings. At least 40 different nations now have some 
form of rankings that are regularly published, with more added each year. The criteria vary significantly; 
the best known and most influential ranking schemes evaluate institutions primarily on the basis of 
academic factors, but some ranking systems use the number of new patents acquired, climate for 
female students, environmentally friendly practices, and a host of other characteristics. 



Ranking tables (referred to as league tables in the United 
Kingdom) serve many purposes, depending on the audience 
and on the nature of the rankers. The organizations that rank 
colleges include media, government agencies, independent 
nonprofits, and academic research centers. These organizations 
have different goals. Some are responding to demands from 
consumers for easily interpretable information on the standing 
of higher education institutions, often stimulating competition 
among those ranked. In some cases, ranking systems are used 
to allocate funding, to highlight distinct missions, or as a means 
of accountability for external audiences. Rankings may include 
only the institutions in a single nation, or they may include 
universities around the world, (box i) 

As the variety and popularity of ranking schemes have increased, 
so have concerns about their impact. Controversy surrounding 
their influence on students, institutions, and governments has 
become a ubiquitous topic in higher education literature and the 
mainstream press. Critics say that quantitative rankings do not 
accurately capture an institution’s true strengths and weakness, 
or the quality of education it provides students; that they are 
biased toward certain types of institutions and against others; 
and that the weightings of certain factors are unjustified and run 
counter to the mission of higher education. Much research has 
been conducted on the subject, giving stakeholders ammunition 
to use in pushing for change or developing rankings that best 
meet their goals. Rankers, critics, academic researchers, and 
other interested parties have begun to collaborate informally in 
the International Rankings Expert Group (IREG), leading to the 
development of a set of principles to guide best practices in the 
use of rankings. 1 



1 The Berlin Principles of Ranking in Higher Education and other IREG information are available 
online through the IHEP Ranking Systems Clearinghouse at www.ihep.org/research/rankingsys- 
temsclearinghouse.cfm. 



Ranking systems have both positive and negative effects, 
and understanding the full impact of rankings is no small task. 
Rankings are used in many different ways by a wide variety of 
stakeholders in the academic enterprise, including students, 
institutions, government, and the public. 

This brief addresses one piece of the equation: how rankings 
influence decision making at institutions, including how they 
are used and what types of decisions they drive. Although 
much of the research on this issue pertains to the United States, 
especially to the U.S. News and World Report (U.S. News) 
rankings, evidence suggests that many of the same trends and 
impacts are occurring around the world. For example, a recent 
study on the impact of rankings on institutions globally found 
that 63 percent of responding institutions were taking “strategic, 
organizational, managerial, or academic actions” in response 
to rankings; only 8 percent indicated that they had taken no 
action. 2 The brief looks at how institutions in four countries — 
Australia, Japan, Germany, and Canada — use rankings in 
their institutional decision-making processes. It also includes 
a review of the relationship between rankings and institutional 
decision making, to frame the issue and illustrate how these 
four countries leverage rankings to improve campus practices. 

Methodology 

This study relied heavily on qualitative data-collection methods to 
examine the effects of rankings on institutional decision making. 
Ellen Hazelkorn, director of research and enterprise, dean of the 
Graduate Research School, and director of the Higher Education 
Policy Unit at the Dublin Institute of Technology, and Patti McGill 
Peterson, senior associate at IHEP — conducted semi-structured 
onsite interviews and focus groups. 3 To gauge the influence of 

2 E. Hazelkorn, “Impact and Influence of League Tables and Ranking Systems on Higher Education 

Decision-Making,” Higher Education Management and Policy 19(2) (2007): B7-110. 
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rankings on different institutional structures, the researchers 
included various higher education constituencies at more than 
20 institutions in the four countries. The schools represent an 
array of institution types, from competitive research universities 
to small colleges. IHEP secured consent to participate in the 
study from each institution’s president, and many of the site visits 
were orchestrated with the assistance of an institution’s Office of 
Institutional Research. To encourage candid participation in the 
study, institutions and respondents are anonymous. 

Interviews and focus groups were conducted between January 
and August 2008. Unless otherwise noted, all quotes in the text 
are from those interviews. Questions and discussion points were 
used to investigate how ranking systems affected four broad 
categories: academic standards and decisions, management 
decisions, institutional strategic development, and organizational 
culture. A wide range of campus constituencies participated, 



3 Hazelkorn: Australia, Germany, Japan; Peterson: Canada. 



including senior administrators, planning and marketing 
personnel, faculty and academic staff, and students. The 
participation of diverse stakeholders allowed researchers to 
probe deeply into varying perspectives on a controversial 
topic. The interviews and focus groups were summarized 
for each case study, to develop some overarching findings. 

It is difficult to separate the impact of rankings on institutional 
decision making from the general impact of competitive forces 
and other factors in higher education. This brief reflects the 
perceptions of those interviewed regarding the role of rankings 
in institutional decision making on their campuses; it does not 
attempt to disaggregate the influence of rankings from other 
institutional variables. This qualitative analysis is meant to 
provide a better understanding of how campus stakeholders 
view rankings, the role rankings play in the interviewees’ work, 
and the perceived impact of rankings on institutional planning 
and priories. ^ 



BOX 1 



Understanding the Ranking Systems Landscape 



Ranking approaches and systems, like higher education 
institutions, vary extensively and are often tied to the unique 
higher education context of a given nation. In general, however, 
each system or approach tends to include a similar, logical set 
of elements. First, data are collected, either from existing sources 
or from original surveys. Then variables are selected from the 
information gathered. Next, the indicators are standardized 
and weighted from the selected variables. Finally, in most cases, 
calculations are performed and comparisons made so that 
institutions can be sorted into rank order. 

In a national ranking system, all or nearly all of a country’s 
universities are measured against one another. This was the 
original university ranking format, pioneered by U.S. News 
in 1981 and widely copied in other countries. In most cases, 
all universities in a country are compared; although in some 
cases — such as Canada ( Maclean's ) and the United States 
(U.S. News) — the universities are grouped according to certain 
institutional characteristics and compared within those groups. 

Global institutional ranking systems are a variation on national 
rankings. The two most prominent systems are the Academic 
Ranking of World Universities from Shanghai’s Jiao Tong 
University (SJT), 4 first released in 2003, and the OS World 
University Rankings from the Times Higher Education 



Supplement of Britain (THES), 5 first released in November 2004. 
Because of the difficulty of comparing data across countries, 
both global ranking systems rely more heavily on research output 
and student data than do national rankings. The TFIES ranking 
system makes substantial use of online surveys to determine its 
“Academic Peer Review” score. Other factors include employer 
reviews, faculty citations, and international reputation. The SJT 
ranking system is quite different, focusing on research and using 
purely quantitative data such as the number of faculty and alumni 
receiving Nobel prizes or Fields medals, as well as the number of 
faculty with published or cited work. 

International ranking systems have received even more criticism 
than national systems for ranking universities by a standard- 
ized criteria, an evaluation technique that discounts the wide 
differences in educational practices and goals among nations. 
Nonetheless, in many parts of the world, these global rankings 
are assumed to accurately reflect a country’s standing in 
worldwide competition. 6 



4 See www.arwu.org/rank2008/EN2008.htm. 

5 See www.topuniversities.com/worlduniversityrankings/. 

6 Based, in part, on excerpts from the 2007 IHEP report, College and University Ranking Systems: 
Global Perspectives and American Challenges. The full report is available online at www.ihep.org/ 
assets/files/publications/a-f/CollegeRankingSystems.pdf. 
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